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Introduction 

A common  parameter  for  describing  the  accuracy  of  a weapon  is  the  circular 
probable  error,  generally  referred  to  as  CEP.  CEP  is  simply  the  bivariate  analog 
of  the  probable  error  of  a single  variable  and  measures  the  radius  of  a mean 
centered  circle  which  includes  50%  of  the  bivariate  probability  mass.  In  the  case 
of  circular  normal  errors  where  the  error  variances  are  the  same  in  both  directions, 
CEP  can  be  expressed  in  terms  of  the  common  standard  deviation,  and  estimators  are 
easily  formulated  and  compared.  In  the  case  of  elliptical  normal  errors,  CEP  can- 
not be  expressed  in  closed  form,  and  hence,  estimators  are  less  easily  formulated. 
Hie  problem  addressed  herein  is  the  comparison  of  CEP  estimators  for  the  elliptical 
case  based  on  some  of  the  commonly  used  CEP  approximations. 


It  will  be  instructive  to  first  review  the  case  of  circular  normal  errors. 


In  general,  it  will  be  assumed  that  the  errors  in  the  X and  Y directions  are 

2 2 

independent  with  mean  zero  and  variances  o and  o , respectively.  Under  the 

x y 

2 2 2 

circular  normal  assumption,  a = a = a and  the  bivariate  distribution  of 

x y 

errors  is  given  by 


2iro 


- “ < x,y  < 00 . 


(1) 
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2 2 H 

The  distribution  of  R - (X  + Y ) ‘ is  easily  derived  and  found  to  be 


gc(r)  s ■$?  e"r  , r > 0 . 
c 


(2) 


This  is  the  well  known  Rayleigh  distribution  with  cumulative  distribution 
function 


P(R  < r) 


C3) 


By  definition,  Gc(CEP)  s .5,  and  the  solution  of  equation  (3)  yields  the  well- 
known  expression 


CEP  = C-22n  (.S0)]!i  a = 1.1774 a . 


(4) 


Four  estimators  for  CEP  in  the  circular  case  were  formulated  and  compared  by 
Moranda  (1959) . 

Consider  now  the  case  of  elliptical  normal  errors.  Here  the  variances  are 
unequal,  and  the  bivariate  distribution  of  errors  is  given  by 


fE(x,y) 


1 - h [(x/0  )2  + (y/o )2] 

e x y ’ 


® < x,y  < °°  . 


CS) 


For  this  case,  the  distribution  of  the  radial  error  R was  derived  by  Chew  and 
Boyce  (1961)  and  has  form 

%Cr)  s fV  e’ar  (6) 

x y 

where 


2 2 

o + o~ 
= JL * 

(2a  a )‘ 
v x y 


2 

a 

- JL 


(2a  a )' 
v x yJ 
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and  I is  the  modified  Bessel  function  of  the  first  kind  and  zero  order,  i.e., 

I (X)  .1  I e'x  cos  6 d 6 . 

O 7f  o 


The  cumulative  distribution  function  for  R is  denoted  by 


r 

P(R  < r)  = GE(r)  = $ gE(t)dt,  C 7) 

but  it  cannot  be  expressed  in  closed  form.  Hence,  the  radius  of  the  50%  circle 
for  the  elliptical  case  cannot  be  expressed  by  a simple  formula  as  it  was  in  the 
circular  case.  One  has  to  solve  Gg(CEP)  = .5  by  niraerical  methods  or  by  referring 
to  tables  prepared  by  Harter  (1960),  DiDonato  and  Jamagin  (1962),  and  others.  To 
avoid  using  these  tables  or  numerical  procedures  for  CEP  evaluation,  a number  of 
approximations  have  been  developed  over  the  years.  Five  of  these  approximations 
have  been  chosen  for  examination.  They  are  designated  below  as  CEP^  through  CEP^: 


CEP1  = 
CEP2  = 


2 J, 

a+o 


C8) 

a) 

\h  (ol  + olY 

(9) 

/ 


(11) 


CEP.  - .565  a + .612  a . , cr  .^/a  > .25 

4 max  min’  mm  max 


.667  a + .206  a . , a . /a  < .25 
max  min’  min'  max 


CEP^  and  CEP 2 were  taken  from  Groves  (1961) , CEP^  was  established  by  Grubbs  (1964), 
CEP^  is  a piece-wise  linear  combination  of  the  standard  deviations,  and  CEP^  was 
also  established  by  Grubbs  (1964)  using  a Wilson-Hilferty  transformation  of  the 
chi-square  in  CEP^.  Plots  of  each  approximation  versus  the  true  CEP  as  a function 
°mii/amax  are  5^own  Figures  1 through  5.  These  give  a fairly  good  indication 
of  how  well  each  performs.  It  is  seen  that  CEP^  deteriorates  rapidly  as  we  depart 
from  the  circular  case  (for  which  CEP-^  degenerates  to  1.1774a),  CEP2  is  reasonably 
good  if  the  ratio  amin/amax  is  not  less  than  about  .2,  CEP^  appears  good  for  all 
ratios,  and  CEP^  and  CEPS  appear  good  to  a lesser  extent  for  all  ratios. 

If  these  approximations  were  used  only  as  approximations  for  assumed  values 
of  the  error  variances  (as  one  does  in  wargaming  and  round  requirement  studies) , 
then  there  would  be  no  estimation  problem.  However,  in  many  cases,  weapons 
analysts  are  using  estimates  of  the  variances  in  these  approximations  (based  on 
sample  data)  to  form  estimates  of  CEP.  Hence,  the  problem  now  becomes  an  estima- 
tion problem  instead  of  an  approximation  problem.  In  particular,  the  problem 
addressed  in  this  paper  is  that  of  comparing  the  five  estimators  for  CEP  formed 

by  replacing  the  population  variances  in  equations  (8)  through  (12)  with  sample 
2 2 2 2 

variances  Sx  = E X^/n  and  = £ Y^/n.  (In  these  expressions,  and  are  the 
recorded  errors  in  the  X and  Y directions,  respectively,  for  the  ith  impact  and 
n is  the  number  of  sample  impacts.)  These  estimators  will  be  referred  to  as 

A At 

CEP.  through  CEPc  in  the  discussion  which  follows. 


Methodology 


Measures  of  comparison  employed  in  this  study  were  the  mean  squared  error 
(MSB) , expected  confidence  interval  length,  said  confidence  interval  confidence. 

With  regard  to  the  former,  the  MSE  of  an  estimator  6 for  a parameter  G is  defined 
in  the  usual  sense,  i.c., 

MSE(O)  = E(6  - 8) 2 « V(G)  + B2  (0) 

where  P,  represents  expectation,  V represents  variance  and  B represents  bias.  It 
was  chosen  because  it  accounts  for  bias  as  well  as  variance  and  all  five  estimators 
arc  biased  for  CEP  except  in  the  degenerate  circular  case.  With  regard  to  the 
second  measure,  it  was  chosen  because  it  too  accounts  for  bias  as  well  as  variance 
but  in  the  sense  of  interval  estimation  vice  point.  These  computations  were  based 
on  approximate  distributions  of  CEP  estimators  and  did  not  provide  intervals  with 
specified  confidence  in  all  cases.  Hence,  the  third  measure  was  included  to 
estimate  the  true  confidence. 

The  computation  of  these  measures  was  straightforward  but  not  simple  due  to 
the  complexity  of  the  estimators.  Recall  that  1,  3 and  5 each  involve  radicals 
of  linear  combinations  of  sample  variances  and  estimators  2 and  4 involve  linear 
combinations  of  sample  standard  deviations.  Hence,  the  sampling  distributions 
were  approximated.  The  approximations  were  achieved  by  matching  the  variance  of 
estimators  3 and  5 with  the  variance  of  the  chi-square  distribution  and  by  match- 
ing the  variance  of  estimators  2 and  4 with  the  variance  of  the  chi  distribution. 
Estimator  1 was  simply  approximated  by  a chi-square  with  2n  degrees  of  freedom. 

’Hi is  distribution  is  exact  only  in  the  circular  case  and  was  included  to  show  how 
poorly  it  becomes  when  eccentricity  of  the  distribution  increases.  The  approxi- 
mations arc  shown  in  Figure  6 and  are  discussed  in  more  detail  in  the  next 
paragraph. 
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Figure  6 provides  a summary  of  approximate  distributions  for  each  CEP^ 
and  defines  several  multiplicative  factors,  JC  , eccentricity  c and  degrees  of 
freedom  v and  v' . v*  does  not  have  a simple  form  and  is  described  below. 

Because  estimators  3 and  S are  of  the  same  general  form,  the  distribution 
of  the  squares  of  both  was  approximated  by  matching  the  variance  of 


V (S^  ♦ s2) 

(a£  + a*) 
v x yJ 


(13) 


with  2 v1 , the  variance  of  a chi-square  with  v'  degrees  of  freedom.  It  was  found 

2 2 

that  v'  = nv  where  v * — . ■ - — - . Expression  (13)  can  be  rewritten  as 

c + 1 


A 2 
u’  cep: 

— — - — where  i = 3 or  5 (14) 

$ 

to  conform  to  the  expressions  in  Figure  6. 

Estimators  2 and  4,  representing  linear  combinations  of  the  standard 
deviations,  were  approximated  by  matching  the  variapce  of  a chi  with  v*  degrees 
of  freedom  with  the  variance  of 


(v*)h  (Sx  + Sy) 

a + o 
x y 


The  variance  of  expression  (15)  was  found  to  be 


v*  (1  - H2 (n) ) 1+c 


(l+c)‘ 


and  the  variance  of  a chi  with  v*  degrees  of  freedom  is 


(15) 


(16) 


v*(l  - H2(v*)) 
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where  H(x) 


■>/r 


rtx) 

r(5) 


Upon  equating  the  two,  we  find  that  v*  satisfies 

H(v*)  - 


1.(1.  H2(n))  -±£Z~- 

a+cr 


and  can  be  obtained  from  a table  of  inverse  solutions  of  the  H function. 

The  approximate  distributions  allow  one  to  derive  approximate  mean  squared 

A 

errors  for  the  CEP^  estimators  which  are  given  in  Figure  7.  The  coefficients 
are  defined  as  before  and  Bias (CEP/)  is  defined  as 

E(CEPi)  - True  CEPi  for  i = 1,  2 5. 


In  general,  a is  a scale  factor  representing  the  maximum  a value;  however , in 

A A 

the  examples  given  here  o^  is  always  equal  to  1.  Note  that  M5E(CEP2)  and  MSEfCEP^) 
can  be  expressed  in  exact  rather  than  approximate  form. 

Since  a point  estimate  may  not  provide  adequate  information,  approximate 
9So  confidence  intervals  were  constructed  for  each  estimator  using  the  distributions 
discussed  above.  The  approximate  100 (1- a) % confidence  limits  for  CEP  are  given 
by 


CEPi 


CEP, 


(V^.l-a/a^i)  (xvi,ot/2^vi) 


where  CEP^  is  the  ith  estimator  and  equals  the  degrees  of  freedom  associated 
with  CEIT . Expected  confidence  interval  widths  can  then  be  computed  and  used  as 
measures  of  comparison  between  estimators.  Clearly,  if  one  could  compute  exact 
95%  confidence  intervals,  comparison  of  interval  widths  would  be  straightforward. 
However,  only  approximate  intervals  can  be  obtained  and  the  confidence  associated 
with  each  interval  must  be  computed  before  a complete  evaluation  can  be  made. 
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Confidence  was  estimated  using  10,000  Monte  Carlo  replicates  for  samples 
of  size  5,  10  and  20  and  measuring  the  percentage  of  time  the  true  CEP  fell 
within  the  interval.  Confidence  and  expected  confidence  interval  widths  were 
then  jointly  examined, 

Results 

The  object  of  this  study  was  to  examine  Mid  evaluate  the  behavior  of 
several  candidate  CEP  estimators  over  a wide  range  of  conditions.  Sanple  sizes 
ranged  from  5 to  400  and  eccentricities  ranged  from  c - 1,  the  circular  case  to 
c = 20,  a highly  elliptical  case.  Extreme  values  of  the  sample  size  and 
eccentricity  may  be  infrequently  encountered  but  were  included  for  completeness. 
Clearly,  an  estimator  behaving  poorly  under  circumstances  unlikely  to  be  observed 
should  not  be  disregarded  as  a viable  candidate. 

Prior  to  determining  approximate  distributions  and  mean  squared  error 
(MSE)  approximations  for  the  estimators,  a Monte  Carlo  simulation  was  developed 
for  computing  the  variance,  bias,  average  squared  error  (ASE)  and  standard  error 
for  each  estimator  at  each  of  three  sample  sizes  (n  = 5,  10,  20).  The  simulated 
ASE's  were  used  as  a check  against  MSE  approximations  which  were  subsequently 
computed. 

Upon  comparing  the  simulated  ASE's  against  results  of  the  NEE  approximations 
for  sample  sizes  S,  10,  and  20,  it  became  evident  that  MSE  approximations  were 
inadequate  for  estimators  3 and  5.  In  fact,  in  the  mid-range  of  the  eccentricity, 
c,  the  NEE  for  3 and  S differed  from  the  simulated  values  of  ASE  by  as  much  as 
three  times  the  standard  error.  For  this  reason,  the  simulated  ASE  values  are 
presented  in  Figure  8 while  the  approximate  MSE  values,  found  suitable  for 
larger  sample  sizes,  are  shown  in  Figure  9, 

Despite  some  fluctuation  at  c 3 .05,  Figures  8 and  9 show  estimators  2 
through  5 producing  fairly  close  results.  As  the  sanple  size  increased, 
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estimator  4 exhibited  the  smallest  mem  squared  error  and  appeared  to  be  the 
most  satisfactory  point  estimator. 


Figures  10  and  11  contain  expected  confidence  interval  width  and  confidence 
interval  confidence,  respectively.  If  the  computation  of  exact  95-d  confidence 
intervals  were  possible,  a straightforward  selection  of  the  estimator  producing 
the  narrowest  width  could  be  made.  However,  the  approximate  confidence  intervals 
have  varying  levels  of  confidence  associated  with  them,  all  of  which  under- 
estimate or  overestimate  the  desired  951  level.  It  appears  that  the  wider  lengths  j 
are  associated  with  higher  confidence  and  the  narrower  widths  with  the  lower 

i 

confidences  so  that  a true  comparison  is  not  really  possible.  However,  it  is 
evident  that  estimators  2 through  5 do  not  distinguish  themselves  as  being  far  I 

superior  or  grossly  inferior  to  one  another.  This  is  essentially  the  same  result 
obtained  from  the  MSE  comparisons . 

In  summary,  unless  c is  very  small,  estimators  2 through  5 produce 
reasonably  close  results.  If  confidence  intervals  are  not  desired,  estimator  4 
would  be  an  acceptable  choice.  Otherwise,  estimator  3 is  recommended  due  to 
ease  of  confidence  interval  computability. 
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In/C  mo* 


32. 


MEAN  SQUARED  ERRORS 


MSE (CEP, j S o-2  K, 

MSE  (CEPJ  = a2  k; 

MSE  (CEPj)  « or*  Kj 

v'  - nu  v 

MSE  (CEPJ  = o2  ( g 2 +g2  c2) 
MSE  (CEP5|  ~ a2  K2  (-^i) 

* Exact 


1-H2  (2n)]  + [Bias (CEP,)]2 
1-Hz(n)]  + [Bias(CEP2)]2 

[l-H2(  ) + [Bias  (CEP3)]2 

= [ciiil2 
c4+  1 

1— H2(n(]  + [Bias  (CEPj]2 
1-H2|r)]+ [Bias  (CEPJ 

-e  7 
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AVERAGE  SQUARED  ERROR 

N - 5 


c 

ASE  1 

ASE  2 

ASE  3 

ASE  4 

ASE  5 

S.E. 

1.0 

.068 

.069 

.070 

.069 

.069 

.003 

.75 

.056 

.053 

.053 

.053 

.053 

.002 

.50 

.056 

.042 

.041 

.041 

.042 

.002 

.35 

.063 

.037 

.038 

.038 

.039 

.002 

.20 

.073 

.035 

.039 

.040 

.041 

.002 

.05 

.079 

.041 

.044 

.043 

.045 

.002 

N = 10 


C 

ASE  1 

ASE  2 

ASE  3 

ASE  4 

ASE  5 

S.E. 

1.0 

.034 

.035 

.035 

.035 

.035 

.002 

.75 

.029 

.027 

.027 

.027 

.027 

.002 

.031 

.021 

.021 

.021 

.021 

.001 

.35 

.038 

.019 

.019 

.019 

.020 

.001 

.048 

.018 

.020 

.021 

.021 

.001 

.053 

.022 

.022 

.022 

.023 

.001 

N = 20 


C 

ASE  1 

ASE  2 

ASE  3 

ASE  4 

ASE  5 

S.E. 

wm 

.017 

.017 

.017 

.017 

.017 

.001 

.75 

.014 

.013 

.013 

.013 

.u!3 

.001 

.017 

.011 

.010 

.010 

.010 

.001 

.35 

.024 

.010 

.009 

.009 

.010 

.001 

.034 

.009 

.010 

.011 

.011 

.001 

.039 

.013 

.011 

.011 

.001 

Figure  8 
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MEAN  SQUARED  ERROR 


N = 50 


c 

MSE  1 

MSE  2 

MSE  3 

MSE  4 

MSE  5 

1.0 

.0069 

.0069 

.0069 

.0069 

.0070 

.75 

.0055 

.0054 

.0057 

.0054 

.0057 

.50 

.0077 

.0044 

.0051 

.0042 

.0052 

.35 

.0143 

.0041 

.0049 

.0037 

.0050 

.20 

.0235 

.0036 

.0047 

.0036 

.0050 

.05 

.0275 

.0072 

.0045 

.0042 

.0048 

N 

= 100 

C 

MSE  1 

MSE  2 

MSE  3 

MSE  4 

MSE  5 

1.0 

.0035 

.0035 

.0035 

.0035 

.0035 

.75 

.0029 

.0027 

.0028 

.0027 

.0029 

.50 

.0057 

.0023 

.0026 

.0021 

.0026 

.35 

.0126 

.0022 

.0024 

.0019 

.0026 

.20 

.0220 

.0018 

.0023 

.0018 

.0026 

.05 

.0281 

.0053 

.0023 

.0021 

.0025 

N 

= 400 

C 

MSE  1 

MSE  2 

MSE  3 

MSE  4 

MSE  5 

1.0 

.0009 

.0009 

.0009 

.0009 

.0009 

.75 

.0009 

.0007 

.0007 

.0007 

.0007 

,50 

.0042 

.0007 

.0007 

.0005 

.0007 

.35 

.0113 

.0008 

.0006 

.0005 

.0007 

.20 

.0209 

.0005 

.0006 

.0005 

.0008 

.05 

.0251 

.0039 

.0006 

.0005 

.0007 

Figure  9 
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CONFIDENCE  INTERVAL  LENGTHS 


N = 5 


c 

CL  1 

CL  2 

CL  3 

CL  4 

CL  5 

1.0 

1.213 

1.160 

1.305 

1.145 

1.317 

.75 

1.071 

1.028 

1.175 

1.014 

1.186 

.50 

.950 

.923 

1.118 

.918 

1.131 

.35 

.897 

.889 

1.129 

.917 

1.144 

.20 

.856 

.876 

1.149 

.985 

1.168 

.05 

.841 

.912 

1.176 

1.054 

1.196 

N 

= 10 

C 

CL  1 

CL  2 

CL  3 

CL  4 

CL  5 

1.0 

.792 

.776 

.817 

.768 

.823 

.75 

.698 

.685 

.735 

.676 

.741 

.50 

.622 

.614 

.697 

.601 

.705 

.35 

.587 

.586 

.693 

.586 

.702 

.20 

.564 

.573 

.694 

.636 

.706 

.05 

.553 

.579 

.695 

.662 

.707 

N 

= 20 

C 

CL  1 

CL  2 

CL  3 

CL  4 

CL  5 

1.0 

.537 

.533 

.545 

.531 

.549 

.75 

.475 

.472 

.492 

.467 

.496 

.50 

.423 

.421 

.465 

.411 

.470 

.35 

.400 

.401 

.460 

.393 

.466 

.20 

.384 

.388 

.455 

.429 

.462 

.05 

.378 

.387 

.453 

.441 

.461 

Figure  10 
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SIMULATED  CONFIDENCE  LEVELS 


N = 5 


c 

PROS  1 

PROB  2 

1.0 

.950 

.947 

.75 

.941 

.947 

.50 

.894 

.941 

.35 

.830 

.937 

.20 

.753 

.932 

.05 

.714 

.930 

N 

C 

PROB  1 

PROB  2 

1.0 

.947 

.945 

.75 

.935 

.944 

.50 

.876 

.941 

.35 

.789 

.939 

.20 

.689 

.939 

.05 

.640 

.931 

N 

C 

PROB  1 

PROB  2 

1.0 

.952 

.952 

.75 

.938 

.951 

.50 

.858 

.945 

.35 

.724 

.942 

.20 

.567 

.946 

.05 

.506 

.912 

PROB  3 

PROB  4 

PROB  5 

.963 

.946 

.963 

.965 

.945 

.965 

.968 

.944 

.967 

.963 

.939 

.961 

.952 

.923 

.950 

.950 

.936 

.948 

10 

PROB  3 

PROB  4 

PROB  5 

.955 

.944 

.955 

.958 

.943 

.959 

.967 

.943 

.966 

.967 

.941 

.965 

.959 

.931 

.955 

.951 

.943 

.948 

20 

PROB  3 

PROB  4 

PROB  5 

.956 

.951 

.956 

.960 

.951 

.960 

.970 

.948 

.969 

.970 

.947 

.968 

.961 

.937 

.955 

.950 

.946 

.946 

Figure  11 
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